Imaging Device and Method for Reading Signals From Such Device

ABSTRACT

Each pixel cell ( 12 ) of an image sensor ( 10 ) is made of a 4-Tr structure, in which only one Tr for resetting a column (X) is so added to an ordinary 3-Tr APS as to reset only an arbitrary pixel selectively, thereby to confine the pixel size. When a pixel signal is to be read, the period, for which the pixel signals composing an ordinary image of one frame are read, is finely divided so that the pixel signals of the pixels receiving an ID light for the period are read out bit by bit and repeatedly. At this time, for only the column being read, an electric current is fed to a read amplifier in the pixel cell ( 12 ) or a variable gain amplifier in an output unit ( 14 ), thereby to suppress the power consumption. As a result, a lower power consumption and a higher pixel formation can be attained in an image pickup device for picking up an image and for acquiring the ID information of a light beacon existing in the image pickup range.

TECHNICAL FIELD

The present invention relates to an imaging device having the functionsof capturing an image of an object and receiving an information lightblinking or intensity-modulated at a frequency higher than a standardframe frequency within the imaging area. The present invention alsorelates to a method of reading signals from such an imaging device.

BACKGROUND ART

In recent years, a new type of system called the identification (ID)recognition camera system (or ID cam) has been proposed. It includes anoptical beacon for emitting a blinking signal containing ID informationand other items of information relating to electronic devices and acamera with a high-speed image sensor. For example, a system disclosedin Patent Document 1 captures a series of images with an ID-recognitioncamera and outputs them as a scene image. Then, it decodes the blinkingdata of the optical beacon by use of all pixel values to create an IDimage. Practical applications of this ID-recognition camera system havebeen also proposed, such as an audio assist system disclosed in PatentDocument 2 and an automatic photography system disclosed in PatentDocument 3.

The above-described ID-recognition camera system has the function ofdisplaying an image captured with the camera, on which the IDinformation of each of the optical beacons detected within the capturedimage is shown at or near the detection point of each optical beacon,allowing users to select some of the ID information according tonecessity and to use the related information. Such a system enablesusers to select one of the electronic devices and indicators and performdata communications with the selected device or control it, using ahandheld information terminal.

On the other hand, various techniques using visible light forinformation transmission and communication have been drawing people'sattention. Some of the new approaches are aimed at data communicationusing indoor or outdoor lighting, traffic signals, indicators ofelectronic devices or similar light emitters as optical beacons (forexample refer to Non-Patent Document 1).

Any system employing such an optical beacon requires a special imagingdevice for extracting identification information transmitted by theoptical beacon present within the captured images, which are normal,two-dimensional images. In general, the signal-reading speed of an imagesensor used in a normal imaging device is determined in conformity tothe standard frame frequency. Typically, the frame frequency is 30 Hz(frames per second: fps). In contrast, the blinking (orintensity-modulating) frequency of an optical beacon depends on thetransmission rate of the identification information. To ensure anadequate amount of information to be transmitted, the blinking orintensity-modulating frequency should be minimally several hundreds Hz,preferably 1 to 100 kHz or higher. Thus, the frequency of the opticalbeacon is much higher than the frame frequency. Therefore, it isdifficult to correctly detect the optical beacon by generally knownmethods for reading pixel signals from normal image sensors.

Conventionally, some image sensors that can be used in theaforementioned type of system have been proposed. For example, a systemdisclosed in Non-Patent Document 2 acquires the ID signals by readingpixel signals at a frame rate of 10 kfps, which is much higher than thestandard frame frequency. This method is suitable to create a high pixeldensity device since it allows the use of a pixel circuit approximatelyidentical to those used in normal image sensors without hardlyincreasing the pixel size. However, this system consumes a considerableamount of electricity; therefore it is necessary to supply a largeramount of current to the circuit components, such as an amplifier forreading the signals, since the operation frequency of the signal-readingcircuit increases with the frame frequency. For example, the powerconsumption of the image sensor disclosed in Non-Patent Document 2 is ashigh as 2 Watts, which is considerably higher than those of normal imagesensors having an equal number of pixels. Incorporating such an imagesensor into a small-sized information device is impractical because itwill significantly shorten the operating time of the device.Furthermore, use of an image sensor consuming so much power makes thedevice difficult to design because the heat-releasing performance mustbe considered. Another problem is that the signal-to-noise (S/N) ratioof the readout signal is low. This is because the readout noise is highdue to the high frequency band of the signal-reading circuit while thesignal level is low due to the shorter period of time for collectingelectric charges.

In a system shown in Non-Patent Document 3, an analogue circuit fordetecting the light intensity modulation component of the ID signaltransmitted by the optical beacon is provided for each pixel so that theidentification information can be extracted within a pixel cellreceiving the light emitted from the optical beacon. This method isadvantageous in that, even though the signal change is minimal, themodulation component can be detected with a high S/N ratio since eachpixel cell has inside a high-gain amplifier. However, this method is notsuitable for creating a high pixel density device because the pixel sizeis considerably large due to the use of more transistors in one pixelcircuit than in a pixel circuit of normal image sensors. Moreover,another factor impeding the creation of a high pixel density device isthat the power consumption per pixel cell is much higher than inconventional cases since electric current must be continually suppliedto the amplifier to operate the analogue circuit inside each pixel cell.This is another factor that impedes the creation of a high pixel densitydevice.

[Patent Document 1] Unexamined Japanese Patent Application PublicationNo. 2003-323239

[Patent Document 2] Unexamined Japanese Patent Application PublicationNo. 2003-345376

[Patent Document 3] Unexamined Japanese Patent Application PublicationNo. 2003-348390

[Non-Patent Document 1] “Kashikou Tsuushin Toha (What is Visible LightCommunication?”), [Online], Kashikou Tsuushin Consohshiamu (VisibleLight Communications Consortium), [Searched on Oct. 15, 2004] Internet<URL: http://www.vlcc.net/about.html>

[Non-Patent Document 2] Miyauchi et al., “Kousoku CMOS Imeeji Sensa WoMochiita Nijigen Soujushinki Ni Yoru Heiretsu Hikari Kuukan Tsuushin NoTeian (Parallel Optical Wireless Communication Using High Speed CMOSImage Sensor”), Shingakugihou (The Technical Report of the Institute ofElectronics Information and Communication Engineers of Japan),CS2004-18, 2004

[Non-Patent Document 3] Oike et al., “Fukugou Genjitukan Ouyou Ni MuketaKousoku-Teikido ID Biikon Kenshutsu Imeeji Sensa (Smart Image Sensorwith High-speed High-sensitivity ID Beacon for Augmented RealitySystem)”, Eijougakkaishi (The Journal of the Institute of ImageInformation and Television Engineers), Vol. 58, No. 6, pp. 835-841, 2004

DISCLOSURE OF THE INVENTION Problem to be Solved by the Invention

To solve the above-described problems, the present invention primarilyintends to provide an imaging device for obtaining identificationinformation from an optical beacon present within an imaging area, inwhich the power consumption by the image sensor and other components issuppressed and its pixels can be densely arrayed. More specifically, thepresent invention proposes a technique for preventing the speed ofreading pixel signals from being much higher than the speed of merelyreading normal image signals, or for preventing the read time per pixelfrom being extremely short, while suppressing the increase in the numberof transistors in each pixel circuit. The present invention alsoprovides a method of reading signals from such an imaging device.

Means for Solving the Problems

Thus, in an imaging device for capturing images of an imaging areawithin which a light-emitter producing an information light whosefrequency is higher than the normal frame frequency is present and forcollecting both image information of the imaging area and identificationinformation transmitted by the information light, the first aspect ofthe present invention provides a method of reading pixel signals from animage sensor having a pixel cell array with pixel cellstwo-dimensionally arranged in the form of a matrix having m rows and ncolumns (where m and n are integers larger than one), which ischaracterized by the following steps:

classifying all the pixel cells into two types: one or more pixel cellsreceiving the information light, which are called the signal-receivingpixel cells, and the other pixel cells, which are called the imagingpixel cells;

dividing a first period of time for sequentially reading the pixelsignals from the imaging pixel cells for constructing one frame of imageinto a plurality of fractional time sections, and inserting a secondperiod of time for reading the pixel signal from the signal-receivingpixel cell between each neighboring pair of the time sections; and

reading the pixel signal from the same signal-receiving pixel cell morethan one time during one round of readout of the pixel signals from allthe imaging pixel cells through all the time sections.

The second aspect of the present invention provides an imaging devicefor sequentially reading pixel signals by the method according to thefirst aspect of the present invention, which is characterized by thefollowing elements;

a) a pixel cell array with a plurality of pixel cells two-dimensionallyarranged in the form of a matrix having m rows and n columns (where mand n are integers larger than one), each pixel cell having aphotoelectric conversion element for converting a received light to acharge signal and storing the charge signal;

b) a position information storage section for keeping informationindicating the position of one or more signal-receiving pixel cellsreceiving the information light among all the pixel cells;

c) a column selection controller operating on the basis of theinformation stored in the position information storage section, fordetermining, column by column, whether the pixel cell at a given columnin a row specified within the pixel cell array is a signal-receivingpixel cell, and for generating a column selection information based onthe result of determination, where the column selection informationdecides whether or not the pixel signal should be read from the pixelcell concerned; and

d) an output current controller for preventing an output current fromflowing into the pixel cell at any column that has not been selected forreadout by the column selection controller.

EFFECT OF THE INVENTION

In general, the percentage of the area receiving the information lightwithin the pixel cell array is small. Therefore, among all the m×n pixelcells, the number of signal-receiving pixel cells can be regarded to bemuch smaller than that of the imaging pixel cells. Accordingly, in thesignal-reading method according to the first aspect of the presentinvention, even if the readout of the pixel signal from eachsignal-receiving pixel cell is repeated more than one time within thesecond periods of time inserted between each neighboring pair of thefractional time sections of the first period of time for reading pixelsignals from all the imaging pixel cells to be used for constructing oneframe of image, it will not be necessary to set the effective speed ofreading the pixel signals much higher than in the normal signal-readingmethods. The repetition count of reading the pixel signal from the samesignal-receiving pixel cell during one normal image frame can bedetermined beforehand according to the frequency of the informationlight.

Thus, in the method of reading signals from an imaging device accordingto the first aspect of the present invention, it is possible to capturenormal two-dimensional images and simultaneously extract identificationinformation from an optical beacon or similar information light presentwithin the imaging area without significantly increasing the readingspeed of the pixel signals. As a result, the imaging device can operateat high speed while suppressing an increase in the power consumption.

In a mode of the method of reading signals from an imaging deviceaccording to the first aspect of the present invention, the divisionalunit of the first period of time is one row of the pixel cell array. Inthis mode, the readout of pixel signals from the imaging pixel cells ina given row and the readout of pixel signals from one or moresignal-receiving pixel cells are alternately performed.

In another mode of the method of reading signals from an imaging deviceaccording to the first aspect of the present invention, the divisionalunit of the first period of time corresponds to a fractional section ofone row of the pixel cell array. In this mode, the readout of pixelsignals from the imaging pixel cells in a fractional section of a givenrow and the readout of pixel signals from one or more signal-receivingpixel cells are alternately performed.

The signal-receiving pixel cells from which the pixel signals are to beread during each round of the second periods of time may be those pixelcells located in one row within an information light-receiving area ontowhich one information light is cast. In this case, the readout of thepixel signals from the imaging pixel cells included in the divisionalunit and the readout of the pixel signal from one or moresignal-receiving pixel cells located in one row within an informationlight-receiving area onto which one information light is cast arealternately performed.

As opposed to the previous case, if the number of the signal-receivingpixel cells among all pixel cells cannot be regarded as much smallerthan that of the imaging pixel cells, repeating the readout of pixelsignals from all the signal-receiving pixel cells many times willrequire a higher readout speed. In practice, if the informationlight-receiving area is large, reducing the number of pixel cells fromwhich the pixel signals are to be read within the aforementioned areawill cause no problems because the pixel cells within that area receivethe same content of information. Accordingly, if the number ofsignal-receiving pixel cells within one information light-receiving areais large, it is preferable to reduce the number of pixel cells fromwhich the pixel signal is to be read by skipping a portion of thesignal-receiving pixel cells or integrating the pixel signals of aplurality of the signal-receiving pixel cells. This method prevents anunnecessarily large number of pixel signals from being read from thesignal-receiving pixel cells, thereby suppressing the increase in thereadout speed.

In the imaging device according to the second aspect of the presentinvention, when a readout row is designated in the pixel cell array, ifthe column selection controller selects a column from which the pixelsignal is to be read, the output current controller prevents the outputcurrent from flowing into the pixel cell at any column that has not beenselected for readout. According to this method, a column including asignal-receiving pixel cell receives no output current during the periodof time for reading pixel signals from the imaging pixel cells to obtainimage signals. Conversely, a column including an imaging pixel cellreceives no output current during the period of time for reading pixelsignals from the signal-receiving pixel cells to obtain identificationinformation. When no output current is flowing, the pixel-readingamplifiers in the pixel cells and the output circuit amplifier for thecolumn concerned will be inactive and their power consumption will bevirtually zero. Thus, the electric power is efficiently saved and powerconsumption is suppressed.

In a mode of the imaging device according to the second aspect of thepresent invention, the position information storage section has a memoryarea having a size of one row and n columns or p rows and n columns(2≦p<m) to be associated with one information light-receiving area,where the binary signal level (“High” or “Low”) of one bit of the memoryarea corresponding to one column of the pixel cell array is determinedaccording to whether a signal-receiving pixel cell belonging to theinformation light-receiving area concerned is present within thatparticular column.

This mode reduces the size of the memory area of the positioninformation storage section. Therefore, it is easy to install thisfunction on a chip of the image sensor including the pixel cell array.Particularly, associating p rows of pixel cells with one bit of memoryarea in each column will be effective in decreasing the capacity of thememory area of the position information storage section, although thehorizontal resolution will be lower.

In the imaging device according to the second aspect of the presentinvention, each pixel cell included in the pixel cell array maypreferably include a potential-reset element for a pixel-by-pixelresetting of the electric potential charged in the photoelectricconversion element inside the pixel cell.

This imaging device does not reset the photoelectric conversion elementinside each pixel cell one row at a time but allows any pixel cell in agiven row to be reset anytime. Therefore, the charging periods for thesignal-receiving pixel cells and the imaging pixel cells can beindependently and freely determined.

More specifically, the potential-reset element may include two kinds oftransistors connected in series, one of which is turned on and off by acolumn-reset signal supplied through a column-reset line provided foreach column of the pixel cell array, and the other is turned on and offby a row-reset signal supplied through a row-reset line provided foreach row of the pixel cell array. In other words, each pixel cell isreset by an XY-addressing technique.

According to this construction, an independently resettable pixel cellcan be created by adding only one transistor to a three-transistor (orfour-transistor) circuit used in the pixel circuits of conventional,typical CMOS (complementary metal-oxide semiconductor) image sensors.Therefore, the pixel circuit will not be much larger than those used innormal image sensors. Thus, a high level of pixel density can be easilyachieved.

In the imaging device according to the second aspect of the presentinvention, each column of the pixel cell array may be provided with avariable-gain amplifier for amplifying the pixel signals produced by thepixel cells in the column with a different gain determined according tothe level of the pixel signals.

If the charging period for the signal-receiving pixel cells differs fromthat for the imaging pixel cells, a significant difference willinevitably result between the levels of the pixel signals. If the levelof an input signal is low, the gain of the variable-gain amplifier israised to correct the difference between the level of the pixel signalsproduced by the imaging pixel cells and that of the pixel signalsproduced by the signal-receiving pixel cells. As a result, for example,it will be easier to synthesize a normally captured image and anidentification information image indicative of the position of anoptical beacon.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of the pixel construction of an imagesensor, for explaining the basic principle of the signal-reading methodaccording to the present invention.

FIG. 2 is a schematic diagram of the pixel construction of an imagesensor, for explaining the basic principle of a conventional, typicalmethod of reading image signals.

FIG. 3 is a table roughly showing the relationships between theID-receiving speed and other parameters.

FIG. 4 is a block diagram schematically showing an imaging deviceincluding an image sensor according to an embodiment of the presentinvention.

FIG. 5 is a circuit diagram showing the construction of one pixel cellof the image sensor shown in FIG. 4.

FIG. 6 is a schematic diagram roughly showing the construction of theoutput section in the image sensor shown in FIG. 4.

FIG. 7 is a schematic diagram of one variable-gain amplifier shown inFIG. 6.

FIG. 8 is a block diagram showing the main components involved in theoperation of the imaging device according to the present embodiment.

FIG. 9 is a timing chart roughly showing the process of collecting bothimage signals and ID information in parallel by the imaging deviceaccording to the present embodiment.

FIG. 10 is a detailed timing chart showing the operation of readingpixel signals in the imaging device according to the present embodiment.

FIG. 11 is a schematic general view of a free-space opticalcommunication system employing a method of reading signals from animaging device according to the present invention.

FIG. 12 is a schematic view of an image shown on the display of aninformation terminal used in the system shown in FIG. 11.

FIG. 13 is a graph showing the frequency spectrums of the signalscontained in the light emitted from the optical beacon of eachcommunication node.

BEST MODE FOR CARRYING OUT THE INVENTION

An example of a free-space optical communication system using an imagingdevice according to the present invention is described with reference tothe attached drawings.

FIG. 11 is a schematic view of the present free-space opticalcommunication system. The information terminal 1, which is hand-held bya user (not shown), is a mobile phone capable of free-space opticalcommunication. The personal computer 2, digital camera 3 and portablemusic player 4 are communication nodes capable of sending an ID signal(which corresponds to the identification information in the presentinvention) through an optical beacon to the information terminal 1. Forexample, this optical beacon can be created using a light-emitting diodefor indicating the on/off state of an electronic device.

FIG. 13 is a graph showing the frequency spectrums of the signalscontained in the light emitted from the optical beacon of eachcommunication node. The pilot signal is a signal that blinks (or changesits intensity) at a frequency of f_(p) lower than one-half of thestandard frame frequency (30 Hz). This pilot signal is common to all thecommunication nodes. The main function of the pilot signal is to makethe position of each device recognizable to the information terminal 1.The ID information has a frequency band spread around the centralfrequency f_(ID) of about 1 kHz, which is much higher than the frequencyf_(p) of the pilot signal. This signal contains various items ofinformation, including an inherent address having a long bit length,like an IPv6 address, uniquely assigned to each communication node, andoperational condition data of the device concerned.

To enable the transmission of image data or audio data at high speeds,it is possible to superimpose another signal onto the light emitted fromthe optical beacon within a frequency band higher than the frequency ofthe ID signal. Alternatively, those kinds of data may be transmitted byBluetooth® or other radio communication techniques without using opticalcommunications.

When the user wants to have a data communication with a certaincommunication node or control a certain communication node, for example,he or she directs the camera of the information terminal 1 to thatcommunication node. Then, as shown in FIG. 12, a two-dimensional imageof the space included in the imaging area of the camera is displayed onthe screen of the display 5 of the information terminal 1. In thisstate, the optical beacon of each active communication node within theimaging area is detected by a method to be explained later. Then, inproximity to each optical beacon, identification information 6 (which ishereby a serial number) for selecting the corresponding communicationnode is displayed on the screen. Any communication node having theidentification information displayed can perform a data communicationwith the information terminal 1 or be controlled from the informationterminal 1. The user can select a communication node and sendpredetermined commands to that node by performing predeterminedoperations on the information terminal 1.

The imaging device according to the present invention is built into theinformation terminal 1. It has the function of collecting image signalsfor reproducing the captured image, and ID information originating fromthe optical beacon of each communication node. As will be explainedlater, the imaging device includes an image sensor. One of the importantcharacteristics of this device exists in its method of reading pixelsignals from the image sensor. The basic principle of thissignal-reading method is described below with reference to FIGS. 1 and2.

FIGS. 1 and 2 schematically show the pixel construction of an imagesensor, where FIG. 1 shows the method of reading pixel signals accordingto the present invention, and FIG. 2 shows a conventional, typicalmethod of reading pixel signals. In each of FIGS. 1 and 2, (a) showsserial numbers indicating the order of reading the pixel signals, and(b) shows the same reading order by trajectories (lines).

To simplify the explanation, it is hereby assumed that a total of 64pixel cells 12 are two-dimensionally arrayed in the form of an 8×8matrix having eight columns arranged in the row (X) direction and eightrows in the column (Y) direction. It should be understood that thenumber of pixel cells actually used is much larger than in the presentexample. The column addresses X0 through X7 are assigned to the columns,respectively. Similarly, the rows have the row addresses Y0 through Y7,respectively. The position of each pixel cell is represented by acombination of column (X) and row (Y) addresses, e.g. (X0, Y0) or (X3,Y7). By this notation, any of the pixel cells can be identified. As willbe detailed later, each pixel cell has a photodiode as a photoelectricconversion element. A ray of light cast on the pixel cell array isphotoelectrically converted by each photodiode, whereby each pixel cellindependently produces an electric signal (or pixel signal).

In the conventional, typical signal-reading method, the electric chargeproduced by the photoelectric conversion is accumulated at every pixelfor a predetermined charging period. Then, in the subsequent readoutperiod, the pixel signal is read from each pixel cell in ascending orderof the row addresses and also in ascending order of the column addresseswithin each row. Normally, the frame frequency of the image is 30 Hz.Therefore, the charging period and the readout period are assignedwithin a single frame period of 30 Hz. All the pixel signals are readout during that readout period and processed by an externalimage-processing circuit to reproduce a piece of image. As a result, oneimage is obtained in every frame period, normally at intervals of 33.3milliseconds.

In contrast, the object whose image is to be captured by the imagingdevice according to the present invention includes an optical beaconcontaining ID information. The optical beacon, which may be found at anylocation within the imaging area, is blinking or changing its intensityat a frequency much higher than the frame frequency, within a range fromseveral hundreds hertz to several kilohertz. The position of the opticalbeacon within the captured image can be recognized by the pilot signalsuperimposed in a separate frequency band, as explained earlier. If thispilot signal is blinking at a frequency lower than one half of the framefrequency (preferably, lower than a quarter), the pilot signal can bedetected by calculating the difference between continuously capturedimages or by carrying out a frequency filtering process at each pixel.Based on the pilot signal detected, the area of the pixel cellsreceiving the light emitted from the optical beacon (which is called theID light hereinafter) can be recognized.

As shown in FIG. 1( a), in the present example, there are two differentID lights present within the imaging area. One ID light-receiving areaID#1 spreads over four pixel cells: (X2, Y3), (X3, Y3), (X2, Y4) and(X3, Y4). The other ID light-receiving area ID#2 spreads over four pixelcells: (X5, Y4), (X6, Y4), (X5, Y5) and (X6, Y5).

The signal-reading method according to the present invention includes acharacteristic readout process for obtaining ID information from IDlight in parallel to collecting image signals for reproducing normalimages. In brief, all the pixel cells are divided into two types: thosepresent within the ID light-receiving area (which correspond to thesignal-receiving pixel cells in the present invention), and thosepresent within the area capturing only normal images (which correspondto the imaging pixel cells in the present invention). Then, a pluralityof readout periods for reading pixel signals from the samesignal-receiving pixel cells in a time-sharing manner are arrangedwithin the period of time for reading pixel signals from all the imagingpixel cells for reproducing one frame of image.

In the present example, there are eight signal-receiving pixel cells:the four pixel cells (X2, Y3), (X3, Y3), (X2, Y4) and (X3, Y4) presentwithin the ID light-receiving area ID#1 and the other four pixel cells(X5, Y4), (X6, Y4), (X5, Y5) and (X6, Y5) present within the IDlight-receiving area ID#2. The other fifty-six pixel cells are theimaging pixel cells. The pixel cells are divided into two groups bytheir column addresses: X0-X3 and X4-X7. Now, let the divisional areahaving the column addresses of X0-X3 be denoted by P1, and the otherdivisional area having the column addresses of X4-X7 by P2.

The present readout process follows these instructions:

(1) In each row of each divisional area (P1 or P2), the imaging pixelcells should be continuously read in ascending order of their columnaddresses.

(2) After the readout of the imaging pixel cells in one row of onedivisional area is completed, the signal-receiving pixel cells in onerow of one ID light-receiving area should be read in ascending order oftheir column addresses.

(3) After the readout of the signal-receiving pixel cells in one row ofone ID light-receiving area is completed, the readout point should bemoved from the divisional area containing the imaging pixel cells thathave been previously read to the other divisional area, where thereadout operation should be restored. If the readout of all the imagingpixel cells in one row has been already completed, the readout pointshould be moved to the next row. If the readout of all the imaging pixelcells in one row has not been completed, the operation should restartfrom the remaining section of the same row.

(4) Concerning the readout of the signal-receiving pixel cells, priorityshould be given to the cells belonging to the same ID light-receivingarea. After the readout of the signal-receiving pixel cells in one IDlight-receiving area is completed, the readout point should be moved tothe signal-receiving pixel cells in the next ID light-receiving area.When one readout round of the signal-receiving pixel cells is completedthrough all the ID light-receiving areas, the readout process shouldreturn to the first ID light-receiving area and repeat the same steps.

More specifically, referring to FIG. 1, pixel signals are read from theimaging pixel cells having the row address Y0 in ascending order oftheir column addresses: X0, X1, X2 and X3. Thus, the readout of one rowof the imaging pixel cells in the divisional area P1 is completed. Next,pixel signals are read from two pixel cells (X2, Y3) and (X3, Y3) in onerow of the signal-receiving pixel cells in the ID light-receiving areaID#1, whereby the readout of one row of the signal-receiving pixel cellsin one ID light-receiving area is completed. Then, the readout point nowmoves to the next divisional area P2, where pixel signals are read fromthe imaging pixel cells in the remaining section of the row having theaddress Y0, in the order of their column addresses: X4, X5, X6 and X7.Subsequently, pixel signals are read from two signal-receiving pixelcells (X2, Y4) and (X3, Y4) in the second row of the ID light-receivingarea ID#1.

Thus, the pixel signals produced by the imaging pixel cells and thoseproduced by the signal-receiving pixel cells are sequentially read out.After the readout of the pixel signal from the pixel cell (X3, Y2) iscompleted, the signals are again read from the signal-receiving pixelcells (X2, Y3) and (X3, Y3) in the first row of the ID light-receivingarea ID#1, which have already been read one time. Subsequently,according to the previous instructions, the pixel signals are read fromall the pixel cells, at least one time for each cell. As a result,during the period for reading pixel signals necessary for constructingone frame of image, the pixel signal will be repeatedly read from eachsignal-receiving pixel cell of the same ID light-receiving areas ID#1and ID#2 multiple times.

After the pixel signal is read, the photodiode of the signal-receivingpixel cell is immediately reset and starts to accumulate an electriccharge according to the amount of light received. As a result, thereadout of the pixel signal from each signal-receiving pixel cell in theID light-receiving areas ID#1 and ID#2 and the accumulation of electriccharge will be repeated at shorter intervals of time (i.e. at afrequency higher than the frame frequency). In the example of FIG. 1,the pixel signal is read from each signal-receiving pixel cell in the IDlight-receiving areas ID#1 and ID#2 four times during the period forreading image signals for one frame of image. Thus, it can be said that,as far as the signal-receiving pixel cells are concerned, the pixelsignal is read at an effective frame frequency higher than the framefrequency of normal images.

Since the signal is read four times from one pixel cell from which thesignal was conventionally read only one time within one frame period, itis necessary to set the readout speed per pixel cell higher than in theconventional cases if the image is to be repeatedly captured at thenormal frame frequency, 30 Hz. However, as long as the total number ofpixel cells in the ID light-receiving areas is significantly less thanthe total number of the pixel cells, the increase in the readout speedis adequately small.

From an opposite point of view, this means that, if an IDlight-receiving area is abnormally large or a large number of ID lightsare simultaneously detected, it will be necessary to significantlyincrease the readout speed because there are a considerable number ofthe signal-receiving pixel cells against the total number of pixelcells. The transmission rate of ID information depends on the readoutfrequency of the ID information. In the previous example, thetransmission rate of the ID information increases with the repetitioncount of reading the signal-receiving pixel cells of the same IDlight-receiving area during the period for reading pixel signals for oneframe of image.

In summary, there are tradeoff relations between the transmission rate(or receiving speed) of ID information, the frame frequency of imagesignals, the number of ID lights that can be simultaneously received,the size of the ID light-receiving area, and other factors. The valuesof these parameters should be appropriately selected. FIG. 3 shows roughcalculations of an ID-receiving speed from several parameters withexamples of design values of the imaging device. As will be detailedlater, the signal-reading method according to the present invention iscapable of collecting ID information and image information in parallelwithout significantly increasing the readout speed.

In the previous example, all the pixel cells were divided along thecolumn direction into two divisional areas P1 and P2. The number ofdivisional areas may be larger than two. It is possible to alternatelyread the imaging pixel cells and the signal-receiving pixel cells, rowby row, without defining the divisional areas. For example, the readoutprocess may be as follows: first, the pixel signals of all the imagingpixel cells having the row address Y0 are read; then, the pixel signalsof the signal-receiving pixel cells in the first row of the IDlight-receiving area ID#1 are read; subsequently, the pixel signals ofall the imaging pixel cells having the row address Y1 are read, and soon. It is also possible to assign multiple rows of the signal-receivingpixel cells of the ID light-receiving area to one row of the imagingpixel cells. For example, the readout process may be as follows: first,the pixel signals of all the imaging pixel cells having the row addressY0 are read; then, the pixel signals of the signal-receiving pixel cellsin the first and second rows of the ID light-receiving area ID#1 areread; subsequently, the pixel signals of all the imaging pixel cellshaving the row address Y1 are read, and so on.

Next, an embodiment of the imaging device for carrying out the readoutoperation described thus far is explained. FIG. 4 is a block diagramschematically showing the imaging device including an image sensoraccording to the present embodiment. FIG. 5 is a circuit diagram showingthe construction of one pixel cell of the image sensor. FIG. 6 is aschematic diagram roughly showing the construction of the output sectionin the image sensor. FIG. 7 is a schematic diagram of one variable-gainamplifier used in the image sensor.

The image sensor 10 includes a pixel cell array 11, row decoder 13,output section 14, column decoder 15, ID-mapping table circuit 16 andother components. The pixel cell array 11 has pixel cellstwo-dimensionally arranged in the form of an m×n matrix having n columnsarranged along the column (X) direction and m rows arranged along therow (Y) direction. The values of m and n can be chosen as desired. Inthe present example, n=320 and m=240. The value of m corresponds to thenumber of pixel rows in FIG. 3.

The row decoder 13 has the functions of selecting multiple pixel cells12 in the same row of the pixel cell array 11 through the row-selectionsignal (Row-sel) line B1 and resetting multiple pixel cells 12 in thesame row through the row-reset signal (Row-rst) line B2. The outputsection 14 includes sample-and-hold circuits each provided for eachcolumn of the pixel cell array 11. Pixel signals produced by the pixelcells are sent through the pixel output signal (Pxl-out) lines B4 to theoutput section 14, which outputs those signals one after another throughthe same horizontal output signal (Out) line B9. The column decoder 15controls the output section 14 so that the signals are outputted inseries. The ID-mapping table circuit 16 includes a memory for holdingposition information of an ID light-receiving area in the pixel cellarray 11 and a control circuit for that memory.

Located outside the image sensor 10 is a controller 21 for feedingcontrol signals to each section of the image sensor 10 and a dataprocessor 22 for receiving the series of pixel signals produced by theimage sensor 10 and for performing predetermined data-processing. Usingthe received pixel signals, the data processor 22 reproduces one imageper one image frame from the pixel signals produced by the imaging pixelcells and restores ID information from the pixel signals produced by thesignal-receiving pixel cells. The data processor 22 may be a dedicateddigital signal processor (DSP) or other hardware elements.Alternatively, it is possible to introduce the output signals through aninterface circuit into a computer having a CPU and other components sothat the computer can function as the data processor by performingnecessary operations. A portion of the functions of the controller 21 ordata processor 22 may be installed on the same chip. The point is thatthere is no restriction on the chip structure and other factors whenconstructing the device.

As shown in FIG. 5, one pixel cell 12 includes one photodiode 31 andfour MOS transistors 32, 33, 34 and 35. The anode of the photodiode 31is grounded and the cathode is connected to both the first transistor 32functioning as a row-reset switch and the gate terminal of the thirdtransistor 34 functioning as a source follower amplifier. The firsttransistor 32 is connected through the second transistor 33 to the resetvoltage signal (V-rst) line B5. The gate terminal of the firsttransistors 32 is connected to the row-reset signal line B2. The gateterminal of the second transistor 33 is connected to the column-resetsignal (Clm-rst) line B3. Normally, a direct voltage Vr slightly lowerthan the power-supply voltage Vdd is applied to the reset voltage signalline B5. However, the reset voltage signal line B5 may be connected tothe supply line of the power-supply voltage Vdd.

The source terminal (or output) of the third transistor 34 is connectedthrough the fourth transistor 35 to the pixel output signal line B4. Thegate terminal of the fourth transistor 35 is connected to therow-selection signal line B1. The first transistor 32 functioning as arow-reset switch and the second transistor 33 functioning as acolumn-reset switch are connected in series. Therefore, when both therow-reset signal (Row-rst) and the column-reset signal (Clm-rst) areswitched to “High” level, the photodiode 31 of the corresponding pixelcell 12 is selectively reset. Thus, any of the pixel cells can beindependently reset.

The basic process of photoelectric conversion and signal output by thispixel cell 12 is as follows: when both the row-reset signal (Row-rst)and the column-reset signal (Clm-rst) are switched to the “High” level,the first and second transistors 32 and 33 are turned on. Then, thepotential at the cathode of the photodiode 31 (which is called the“photodiode potential” hereinafter) is reset to the voltage Vr suppliedthrough the reset voltage signal line B5. Under the condition that atleast one of the first and second transistors 32 and 33 is off, if a rayof light falls onto the photodiode 31, a photoelectric current dependingon the strength of the light flows through the photodiode 31. Due to anelectric discharge caused by this current, the photodiode potentialgradually falls. The falling speed of the potential depends on thestrength of the received light. The speed is higher as the light isstronger.

When the fourth transistor 35 is off, this pixel cell 12 is separatedfrom the pixel output signal line B4. If the row-selection signal(Row-sel) supplied to the row-selection signal line B1 is switched tothe “High” level, the fourth transistor 35 turns on. In this state,depending on the photodiode potential at the moment, the current signalflowing to the third transistor 35 functioning as the source followeramplifier can be supplied to the pixel output signal line B4. Whetherthe output current actually flows or not depends on the presence of aload connected to the pixel output signal line B4, as will be explainedlater.

In the pixel cell 12, the source follower amplifier consisting of thethird transistor 34 is basically a simple buffer amplifier. Such a pixelcell 12 can be basically constructed by adding only one row-resettingtransistor to a pixel cell used in a conventional three-transistor typeCMOS image sensor. Its pixel size can be approximately equal to that ofa pixel of a conventional, typical CMOS APS (active pixel sensor) imagesensor. Therefore, the present pixel cells can be closely arranged sothat the pixel cell array 11 can have a large number of pixels withoutincreasing its area.

As shown in FIG. 6, the output section 14 has an output circuit unit 40for each column. The output circuit unit 40 includes a transistor 41functioning as a load current source according to the voltage signal fedthrough the bias voltage signal (Amp-bias) line B6, a load switch 42 forconnecting the transistor 41 as a load to the pixel output signal lineB4 or disconnecting the same transistor from the line B4, avariable-gain amplifier 43, a sampling-and-holding circuit 44 and anoutput-selection switch 50. In each column, the 240 pieces of pixelcells 12 are all connected to the common pixel output signal line B4,through which a current signal is supplied. When the load switch 42 ison, the transistor 41 converts the current signal to a voltage signal.This signal is then amplified by the variable-gain amplifier 43according to necessity and sent to the sampling-and-holding circuit 44.

In the variable-gain amplifier 43, as shown in FIG. 7, a comparator 432or similar circuit element compares an input voltage with apredetermined reference voltage. If the input voltage is equal to orhigher than the reference voltage (or if the pixel output is much lowerthan one sixteenth of the output amplitude), the gain switch 436 isturned off to disconnect the second feedback capacitor 435 and the gainof the amplifier 431 is raised to a level sixteen times the previouslevel. If the input voltage is lower than the reference voltage, thegain switch 436 is turned on so that the capacitance of the inputcapacitor 433 equals the total capacitance of the second feedbackcapacitors 434 and 435, whereby the gain of the amplifier 431 is set toone.

As explained earlier, the pixel signal of the signal-receiving pixelcell is read more frequently than that of the imaging pixel cell.However, its charging time is accordingly short and the level of itspixel signal is low. Therefore, if the level of the pixel signal isadequately low, the signal level is raised by increasing the gain of thevariable-gain amplifier 43 to a level sixteen times the previous level.As a result, the level difference between the pixel signals produced bythe imaging pixel cells and those produced by the signal-receiving pixelcells are made to be as small as possible. Furthermore, saturation ofthe amplifier 431 is prevented, which occurs when the amplitude of theinput voltage is large. Though not shown in the drawings, theinformation indicating the current selection of the gain is stored inassociation with the pixel signals. As will be explained later, whenthose pixel signals are outputted, the gain selection informationassociated with them is also outputted in parallel. The data processor22 uses this gain selection information in the data processing. Anexample of the amplifier 431 is an inverting amplifier, such as asource-common amplifier.

In the present embodiment, the variable-gain amplifier 43 evaluates theinput voltage amplitude by two levels. Alternatively, it is possible toevaluate the amplitude by three or more levels for a finer selection ofthe gain levels. For that purpose, the comparator 432 may includemultiple comparators having different thresholds or multiple levels ofreference voltages may be sequentially inputted into a singlecomparator. In the latter case, the input voltage amplitude can beevaluated by the timing at which the output value of the comparator isinverted.

The sampling-and-holding circuit 44 has two independentsampling-and-holding sections: The first section is for the sampling andholding of image signals and includes an image signal-sampling switch 45and an image signal-holding capacitor 46. The second section is for thesampling and holding of ID signals and includes an ID signal-samplingswitch 47 and an ID signal-holding capacitor 48. The on/off state of theswitch 45 at each column can be controlled through the image-samplingsignal (S/H-img) line B7. The on/off state of the switch 47 at eachcolumn can be controlled through the ID-sampling signal (S/H-id) lineB8. The readout-selection signal (Read-sw) line B10 is used to controlan image/ID selection switch 49, which is used to select either thecapacitor 46 or 48 when the voltage held by the capacitor is to beoutputted.

Each output circuit unit 40 includes an output-selection switch 50,whose on/off state is controlled by an output column selection signal(Clm-sel) given from the column decoder 15 so that the voltage signalheld by either the capacitor 46 or 48 of the sampling-and-holdingcircuit 44 of each output circuit unit 40 is selected one after anotherand sent through the horizontal output signal line B9 to the outside.

As shown in FIG. 8, the ID-mapping table circuit 16 includes a memorycircuit 161 having a memory area for holding position information of oneID light-receiving area, and an ID row decoder 162 and an ID columndecoder 163 for controlling the addressing of the memory area of thememory circuit 161.

In the memory circuit 161, the number of column addresses of the memoryarea is n, which means that this area can hold n bits of information foreach ID light-receiving area (ID light). In the present example, thenumber N of ID lights that can be simultaneously handled is seven.Therefore, the memory area has a capacity of 7×n bits. However, it ispossible to handle more than N ID lights by a technique to be describedlater. One bit of memory area can consist of one delay latch (D-latch).Each bit of the n-bit memory area corresponds to each column of thepixel cell array 11. Each D-latch represents one bit by taking either“High” or “Low” level. The “High” level indicates the presence of asignal-receiving pixel cell in the corresponding column. The “Low” levelshows that all the pixel cells in the column are imaging pixel cells andno signal-receiving pixel cell exists in it.

The memory area of the memory circuit 161 holds only the informationrelating to the positions of the signal-receiving pixel cells in the rowdirection (i.e. the horizontal direction in FIG. 8). The informationindicating the positions of the signal-receiving pixel cells in thecolumn direction (i.e. the vertical direction in FIG. 8) is held in thecontroller 21 external to the image sensor 10. When pixel signals of thepixel cells in a given column are to be read, the information indicatingwhether or not any signal-receiving pixel cell is present in that columnis supplied to the ID row decoder 162 in the form of an ID-selectionsignal (id-sel#1-#7). In practice, it is possible that multiple IDlight-receiving areas exist in the same row (as in FIG. 8, where two IDlight-receiving areas ID#1 and ID#2 are present in the row having therow address Y1). Therefore, N pieces of ID-selection signal lines (inthe example of FIG. 8, there are seven lines) are provided in parallel.An ID-selection signal being at the “High” level indicates that an IDlight-receiving area associated with that ID-selection signal line ispresent in the row being read at the moment.

In the ID-mapping table circuit 16, each column of the pixel cell array11 is checked for an existence of any signal-receiving pixel cell on thebasis of the data pattern in the memory area of the memory circuit 161(which ID light-receiving area includes the signal-receiving pixel cellis not hereby questioned). The check result can be represented by aone-bit signal. A signal being at the “High” level indicates theexistence of at least one signal-receiving pixel cell in the columnconcerned. A “Low” level signal indicates that no signal-receiving pixelcell exists at the column concerned in a row selected by the row decoder13. There will be n pieces of existence-check results, the same as thenumber of the columns. The check result for a given column can beobtained by calculating a logical product (AND operation) of the outputof each D-latch in that column and the ID-selection signal (id-sel#1-#7)corresponding to that D-latch and then calculating a logical add (ORoperation) of all the logical products (N pieces). Such a calculator canbe embodied by a combination of N pieces of AND gate elements and one ORgate element having N inputs or a logical circuit having equivalentfunctions.

The ID-mapping table circuit 16 has two operation modes: normal imagingmode and ID-obtaining mode. The controller 21 selects one of the twooperation modes by a one-bit mode-selection signal (idmt-md). Forexample, the “High” level of the mode-selection signal designates thenormal imaging mode and the “Low” level designates the ID-obtainingmode. The ID-mapping table circuit 16 generates a one-bit column-resetsignal from the existence-check result of the signal-receiving pixelcell and the operation mode. The signal thereby generated is outputtedto the column-reset signal line B3 for the column concerned. If thecolumn-reset signal is at the “High” level, the load switch 42 assignedfor that column in the output section 14 turns on. As a result, thetransistor 41 as the load is connected to, the pixel output signal lineB4, so that a current signal flows from the pixel cell 12. In addition,the switch inside the variable-gain amplifier 43 turns on to allow thecurrent to flow into the circuit. Thus, the amplifier 43 generates anoutput. At the same time, the column-reset signal is also fed to thepixel cell 12.

More specifically, in the normal imaging mode (i.e. mode-selectionsignal=High), if a given column includes a signal-receiving pixel cell(i.e. existence-check result=High), the pixel signal of the pixel cellat the column is not read out (i.e. column-reset signal=Low). If thegiven column includes no signal-receiving pixel cell (i.e.existence-check result=Low), the pixel signal of the pixel cell at thecolumn is read out (i.e. column-reset signal=High). In the ID-obtainingmode (i.e. mode-selection signal=Low), if a given column includes asignal-receiving pixel cell (i.e. existence-check result=High), thepixel signal of the pixel cell at the column is read out (i.e.column-reset signal=High). If the given column includes nosignal-receiving pixel cell (i.e. existence-check result=Low), the pixelsignal of the pixel cell at the column is not read out (i.e.column-reset signal=Low). Such a control can be realized by generating acolumn-reset signal for each column by calculating an exclusive logicalproduct of the one-bit existence-check result and the one-bitmode-selection signal.

Referring to the timing charts of FIGS. 9 and 10 in addition to FIGS.4-8, a typical operation of the imaging device in the present embodimenthaving the construction described thus far is explained. The timingchart in FIG. 9 roughly shows the process of collecting both imagesignals and ID information in parallel by the imaging device accordingto the present embodiment, and the timing chart in FIG. 10 shows thedetails of the operation of reading the pixel signals. It should benoted that, though the standard frame frequency of imaging devices is 30Hz, the frame frequency in the present example is set higher (60 Hz, twotimes the normal value) because a pilot signal of 15 Hz is used todetect the positions of ID light-receiving areas. It is possible tooperate the present imaging device at a frame frequency of 30 Hz.

In the present imaging device, the readout operation is controlled inunits of one frame of normal image: During the vertical blanking timewithin one frame of the normal image (i.e. 16.7 milliseconds in thepresent case), the data held in the memory area of the ID-mapping tablecircuit 16 (i.e. the position information of the signal-receiving pixelcells) are updated. For example, the pilot signal superimposed on the IDinformation is located within the normal image to define the IDlight-receiving area, after which the positions of the signal-receivingpixel cells are determined, as explained earlier. Once the IDlight-receiving area has been defined and the ID information has beencollected throughout the period of time corresponding to one frame ofnormal image, it is possible to re-calculate the position of the centerof gravity of each ID light-receiving area, or the position of thesignal-receiving pixel cell at which the received light is thestrongest, and then update the data held in the memory area of theID-mapping table circuit 16 on the basis of the re-calculated position.

The imaging device whose parameters are designed as shown in FIG. 3 iscapable of repeatedly reading pixel signals of each signal-receivingpixel cells in the same ID light-receiving area 35 times during oneframe of normal image. Since one ID image is created for each readoutprocess, each of up to seven ID light-receiving areas (ID#1-ID#7) willhave 35 pieces of ID images created. The data processor 22 recognizesthe position of each ID light-receiving area from the 35 pieces of IDimages and follows the movement of the center of gravity of the IDlight-receiving area. Then, if necessary, it updates the original dataof the ID-mapping table stored in the controller 21 and also renews thedata held in the memory area of the ID-mapping table circuit 16 duringthe next vertical blanking time.

Updating of the data held in the memory area of the ID-mapping tablecircuit 16, i.e. the position information of the signal-receiving pixelcells, involves the following steps: All the bits of the memory area ofthe memory circuit 161 are reset to the “Low” level at the end of thevertical blanking time of one image frame. Then, an ID-selection signal(id-sel#1-#7) is generated to select one row of the memory area in whichthe data (or bit pattern) are to be changed. Further, a column includingthe signal-receiving pixel cell concerned is selected through theID-column decoder 163. Finally, the “High” level bit is written into thedesignated D-latch. The steps described thus far are repeatedlyperformed for every signal-receiving pixel cell of all the IDlight-receiving areas within the vertical blanking time. FIG. 8 showsthe memory area of the memory circuit 161 into which the bit patterns offour ID light-receiving areas (ID#1-ID#4) have been written.

After the vertical blanking time, when the readout of pixel signals isstarted, the row decoder 13 selects the first row address Y0 and changesthe row-selection signal (Row-sel) for the selected row to the “High”level, as shown in FIG. 10( a). As a result, the fourth transistor 35 ofevery pixel cell 12 in that column turns on. In this state, the currentsignal generated due to the photodiode potential can flow through thethird transistor 34 (i.e. the source follower amplifier) to the pixeloutput signal line B4. At this moment, the mode-selection signal(idmt-md) is set to the normal imaging mode. Therefore, at every columnwhere the pixel cell in the selected row is an imaging pixel cell, theload switch 42 turns on, so that the load current source is connectedand a current is supplied into the circuit of the variable-gainamplifier 43. In contrast, at a column where a signal-receiving pixelcell exists, the output current does not flow and the variable-gainamplifier 43 remains inactive, so that little power is thereby consumed.

The output voltage of the variable-gain amplifier 43 at a column fromwhich a pixel signal has been read out is transferred to the capacitor46 when the image sampling-and-holding signal (S/H-img) is switched tothe “High” level and thereby turns the switch 45 on, as shown in FIG.10( d). This action simultaneously takes place at all the imaging pixelcells in the same row (Y0). Subsequently, the row decoder 13 switchesthe row-reset signal (Row-rst) to the “High” level, as shown in FIG. 10(b). At this moment, the column-reset signal (Clm-rst) for the imagingpixel cells is also at the “High” level. Therefore, the photodiodepotential in the imaging pixel cell is reset to the voltage Vr suppliedthrough the reset voltage line B5. Thus, the sampling-and-holdingoperation of the pixel signal generated by the imaging pixel cells atthe row address Y0 is completed.

Next, under the command of the controller 21, the row decoder 13 selectsthe row address of the first row of one or more rows where the IDlight-receiving area ID#1 exists and switches the row-selection signal(Row-sel) for the selected row to the “High” level. In the case of FIG.8, the row address Y0 is selected. As a result, the fourth transistor 35of every pixel cell 12 in that row turns on. In this state, the currentsignal generated due to the photodiode potential can flow through thethird transistor 34 (i.e. the source follower amplifier) to the pixeloutput signal line B4. At this moment, the mode-selection signal is setto the ID-obtaining mode. Therefore, according to the row-reset signalsfed from the ID-mapping table circuit 16, the load switch 42 turns ononly at each column where a signal-receiving pixel cell exists, wherebythe load current source is connected and a current is supplied into thecircuit of the variable-gain amplifier 43. In contrast, at the majorityof columns in which imaging pixel cells are present, the readout ofpixel signals does not take place and the variable-gain amplifier 43remains inactive, so that little power is consumed.

The output of the variable-gain amplifier 43 at the columns where asignal-receiving pixel cell exists (in the case of FIG. 8, at the columnaddresses X316 and X317) is transferred to the capacitor 48 when the IDsampling-and-holding signal (S/H-id) is switched to the “High” level andthe switch 47 turns on, as shown in FIG. 10( e). This actionsimultaneously takes place at all the signal-receiving pixel cells inthe same row (Y0). Subsequently, the row decoder 13 switches therow-reset signal (Row-rst) to the “High” level, as shown in FIG. 10( b).At this moment, the column-reset signal (Clm-rst) for thesignal-receiving pixel cells is also at the “High” level. Therefore, thephotodiode potential in the signal-receiving pixel cell is reset to thevoltage Vr supplied through the reset voltage signal line B5, asexplained earlier. Thus, the sampling-and-holding operation of the pixelsignals generated by the signal-receiving pixel cells in the first rowof one or more rows where the ID light-receiving area ID#1 exists iscompleted.

In parallel to this sampling-and-holding operation, the process ofreading the pixel signal of the imaging pixel cell previously held bythe capacitor 46 is carried out: As shown in FIG. 10( f), thereadout-selection signal (Read-sw) is set to the “High” level to changethe image/ID-selection switch 49 to the capacitor 46. The column decoder15 turns on one output-selection switch 50 after another by switchingthe output column selection signal (Clm-sel) to the “High” level whilechanging the column address from X0 in ascending order (FIG. 10( g)). Asa result, the pixel signal held by each capacitor 46 is outputted to thehorizontal output signal line B9 one after another. In the presentembodiment, since the pixel cells are divided into five groups along thecolumn direction, the readout process is discontinued once when theround of column addresses X0 through X63 is completed. If asignal-receiving pixel cell is included in this readout range, the pixelsignal at that column is invalid. Therefore, the column address of sucha column may be skipped. Alternatively, it is acceptable to discard theoutput while maintaining the column address (i.e. dummy reading).

Next, the pixel signals of the signal-receiving pixel cells held justbefore by the ID-holding capacitor 48 are read out: As shown in FIG. 10(f), the readout-selection signal (Read-sw) is set to the “Low” level tochange the image/ID-selection switch 49 to the capacitor 48. The columndecoder 15 updates the column addresses where the ID light-receivingarea ID#1 exists in ascending order of the column address, referring tothe information memorized in the ID-mapping table circuit 16 or theinformation written in the controller 21 (FIG. 10( g)). At the columnaddresses selected, the output column selection signal (Clm-sel) is setto the “High” level to turn on the output-selection switch 50. As aresult, the pixel signal held by each capacitor 48 is sequentiallyoutputted to the horizontal output signal line B9. In FIG. 10( g), theID light-receiving area ID#1 is assumed to have s (rows)×r (columns)pixels, so that the r pixel signals are sequentially read out. In FIG.8, two column addresses X316 and X317 are sequentially selected and thepixel signals held by the capacitors 48 at these columns are outputted.

Subsequently, under the command of the controller 21, the row decoder 13selects the row address of the second row of one or more rows where theID light-receiving area ID#1 exists and switches the row selectionsignal (Row-sel) for the selected row to the “High” level. In the caseof FIG. 8, the row address Y1 is selected. Then, the samesampling-and-holding operation as previously performed on the pixelsignals of the signal-receiving pixel cells at the row address Y0 isperformed to transfer each pixel signal to the capacitor 48 at eachcolumn and reset the photodiode potential in the signal-receiving pixelcell at which the readout operation has been completed. In parallel tothis sampling-and-holding operation, the readout process of the pixelsignals of the imaging pixel cells that have been initially held by theimage-holding capacitors 46 is resumed. Since the readout process of thepixel cells in the row address Y0 has been completed up to the columnaddress X63, the operation is continued from the column address X64 inascending order, turning on one output-selection switch 50 after anotherto sequentially output the pixel signals held by the capacitors 46.

By repeating the sampling-and-holding operation and the readoutoperation described thus far, it is possible to extract pixel signalsnecessary for creating one frame of image and, in parallel, repeatedlyread the pixel signals of the signal-receiving pixel cells that arereceiving an ID light. After the imaging signals for one frame of imagehave been collected, the data processor 22 reproduces a piece of imageand also restores ID information.

The embodiment described thus far is a mere example of the presentinvention and can be changed, modified or expanded within the scope ofthe present invention.

For example, to follow a quick movement of ID-receiving pixel cells dueto a camera shake or similar phenomenon, it is possible to individuallyprocess each ID image immediately after reading it rather than merge theID image into one frame of normal image and process the resultant image.In a preferable example, the position of the center of gravity of eachID light-receiving area is calculated every time one ID image is read,and then the position of the pixel cells to be subsequently read foracquiring the next ID image is updated according to the calculationresult.

In the previous embodiment, the photodiode potential of pixel cells wasreset immediately after the pixel signals of those pixel cells had beenread to produce an ID image. However, it is not always necessary toreset it every time the signals are read. In practice, it is unnecessaryto reset the potential until the pixel value is saturated. Even if thepotential is not reset, an ID image can be obtained by calculating thedifferences between the latest pixel values and the previous pixelvalues collected to create the preceding ID image.

In the previous embodiment, the variable-gain amplifier was provided foreach column in order to reduce the noise in a pixel signal having a lowsignal level. The noise can be reduced by different methods. An exampleis a so-called active reset technique, which uses a circuit for reducingthe reset noise of pixel cells for each column of the image cells 11.The circuit generates a feedback signal based on the read pixel valueand feeds it into a resetting transistor provided in the pixel. Thistechnique is effective in reducing the noise of the ID light, whosecharging time is relatively short.

In both normal imaging and ID-obtaining modes, the charging time may beintentionally shortened by shifting the timing of resetting thephotodiode in the pixel cell so as to expand the dynamic range to coverlarger strength values of the incident light. Naturally, shortening thecharging time is disadvantageous for detecting weak incident light.Therefore, it is preferable to add a circuit for adaptively regulatingthe charging time.

In the previous embodiment, the data in the memory area of theID-mapping table circuit were updated during the vertical blanking timewithin one frame of normal image. It is also possible to renew the datain the memory area of the ID-mapping table circuit during an appropriateperiod of time between the vertical blanking times of neighboringframes. For example, suppose that the number of ID light-receiving areasto be simultaneously processed is larger than the row number of thememory area. To deal with such a case, it is possible to repeat thefollowing process during one image frame: the bit pattern in theID-mapping table circuit corresponding to an ID light-receiving area inwhich the pixel signals of the signal-receiving pixel cells have beenread out at least one time is renewed with a new bit patterncorresponding to another ID light-receiving area, and then the readoutof pixel signals of signal-receiving pixel cells is performed accordingto the new bit pattern.

If the readout process of pixel cells is changed, for example if thedivision conditions for continuously reading pixel signals of imagingpixel cells are changed, it is naturally necessary to accordingly changethe control process.

The embodiment can be further changed, modified or expanded in variousforms other than the previous ones within the spirit and scope of thepresent invention, which is clearly defined in the Claims section ofthis patent application.

1. A method of reading pixel signals from an image sensor having a pixelcell array with a number of pixel cells two-dimensionally arranged in animaging device for capturing images of an imaging area within which alight-emitter producing an information light whose frequency is higherthan the normal frame frequency is present and for collecting both imageinformation of the imaging area and identification informationtransmitted by the information light, which is characterized byfollowing steps: classifying all the pixel cells into two types: one ormore pixel cells receiving the information light, which are called thesignal-receiving pixel cells, and the other pixel cells, which arecalled the imaging pixel cells; dividing a first period of time forsequentially reading the pixel signals from the imaging pixel cells forconstructing one frame of image into a plurality of fractional timesections, and determining a readout procedure so that a second period oftime for reading the pixel signal from the signal-receiving pixel cellis inserted between each neighboring pair of the time sections; andreading the pixel signal from the same signal-receiving pixel cell morethan one time during one round of readout of the pixel signals from allthe imaging pixel cells through all the time sections.
 2. The method ofreading signals from an imaging device according to claim 1, which ischaracterized in that the pixel cell array has pixel cellstwo-dimensionally arranged in a form of a matrix having m rows and ncolumns (where m and n are integers larger than one) and a divisionalunit of the first period of time is one row of the pixel cell array. 3.The method of reading signals from an imaging device according to claim1, which is characterized in that the pixel cell array has pixel cellstwo-dimensionally arranged in a form of a matrix having m rows and ncolumns (where m and n are integers larger than one) and a divisionalunit of the first period of time corresponds to a fractional section ofone row of the pixel cell array.
 4. The method of reading signals froman imaging device according to claim 2, which is characterized in that areadout of the pixel signals from the imaging pixel cells included inthe divisional unit and a readout of the pixel signal from one or moresignal-receiving pixel cells located in one row within an informationlight-receiving area onto which one information light is cast arealternately performed.
 5. The method of reading signals from an imagingdevice according to claim 1, which is characterized in that, if thenumber of the signal-receiving pixel cells within one informationlight-receiving area is large, the number of the signal-receiving pixelcells from which the pixel signal is to be read is reduced by skipping aportion of the signal-receiving pixel cells or integrating the pixelsignals of a plurality of the signal-receiving pixel cells.
 6. Animaging device for sequentially reading pixel signals by a methodaccording to claim 1, which is characterized by following elements; a) apixel cell array with a plurality of pixel cells two-dimensionallyarranged in a form of a matrix having m rows and n columns (where m andn are integers larger than one), each pixel cell having a photoelectricconversion element for converting a received light to a charge signaland storing the charge signal; b) a position information storage sectionfor keeping information indicating a position of one or moresignal-receiving pixel cells receiving the information light among allthe pixel cells; c) a column selection controller operating on a basisof the information stored in the position information storage section,for determining, column by column, whether the pixel cell at a givencolumn in a row specified within the pixel cell array is asignal-receiving pixel cell, and for generating a column selectioninformation based on a result of the determination, where the columnselection information decides whether or not the pixel signal should beread from the pixel cell concerned; and d) an output current controllerfor preventing an output current from flowing into the pixel cell at anycolumn that has not been selected for readout by the column selectioncontroller.
 7. The imaging device according to claim 6, which ischaracterized in that the position information storage section has amemory area having a size of one row and n columns or p rows and ncolumns (2≦p<m) to be associated with one information light-receivingarea, where a binary signal level of one bit of the memory areacorresponding to one column of the pixel cell array is determinedaccording to whether a signal-receiving pixel cell belonging to theinformation light-receiving area concerned is present within thatparticular concerned.
 8. The imaging device according to claim 6, whichis characterized in that each pixel cell included in the pixel cellarray includes a potential-reset element for a pixel-by-pixel resettingof an electric potential charged in the photoelectric conversion elementinside the pixel cell.
 9. The imaging device according to claim 8, whichis characterized in that the potential-reset element includes two kindsof transistors connected in series, one of which is turned on and off bya column-reset signal supplied through a column-reset line provided foreach column of the pixel cell array, and the other is turned on and offby a row-reset signal supplied through a row-reset line provided foreach row of the pixel cell array.
 10. The imaging device according toclaim 6, which is characterized in that each column of the pixel cellarray is provided with a variable-gain amplifier for amplifying thepixel signals produced by the pixel cells in the column with a differentgain determined according to a level of the pixel signals.
 11. Animaging device for capturing images of an imaging area within which alight-emitter producing an information light whose frequency is higherthan the normal frame frequency is present, and for at least collectingidentification information transmitted by the information light, whichis characterized by following elements: a) a pixel cell array with aplurality of pixel cells two-dimensionally arranged in a form of amatrix having m rows and n columns (where m and n are integers largerthan one), each pixel cell having a photoelectric conversion element forconverting a received light to a charge signal and storing the chargesignal; b) a position information storage section for keepinginformation indicating a position of one or more signal-receiving pixelcells receiving the information light among all the pixel cells; c) acolumn selection controller operating on a basis of the informationstored in the position information storage section when the pixel signalof a pixel receiving the information light is selectively read, fordetermining, column by column, whether the pixel cell at a given columnin a row specified within the pixel cell array is a signal-receivingpixel cell, and for generating a column selection information based on aresult of the determination, where the column selection informationdecides whether or not the pixel signal should be read from the pixelcell concerned; and d) an output current controller for preventing anoutput current from flowing into the pixel cell at any column that hasnot been selected for readout by the column selection controller.